On the Internet, users actively access WWW servers from Web browsers to browse information such as document data, image data, and the like. In a system environment which includes WWW servers that disclose information on the Internet, and clients used to browse such information in this way, each client can browse information disclosed by the WWW servers using a Web browser.
Each WWW server saves a document called a home page, which is prepared by describing information to be disclosed in HTML. The Web browser on the client side accesses such home page and displays it on a computer of the client side. Also, the Web browser on the client side can acquire required information by tracing links in displayed pages. Furthermore, as a method of downloading files managed by the WWW server, a method called “File Transfer Protocol” (to be abbreviated as “FTP” hereinafter) is known. The FTP is a scheme for transferring the contents of a file on the WWW server to the client computer at a time via a network.
As a protocol that allows the client to fragmentarily access and display an image file managed by the server, Flashpix/IIP is known. This Internet imaging protocol (IIP) is optimal to the image data file format “Flashpix”, and can request partial image data via a network. At this time, access is made for each tile specified by Flashpix. Some conventional techniques that use this IIP protocol have been disclosed (for example, see Japanese Patent Laid-Open No. 2002-49514).
A case will be examined below wherein this Flashpix/IIP is directly applied to JPEG2000. In JPEG2000, encoded data of each scalability is formed of difference data from data of scalability one level lower than that scalability. Hence, a method of cashing fragmentary encoded data received on the client side, and passing all encoded data to a decoder to decode them from the beginning, and a method of stopping a decoder during its process, passing the currently received encoded data to the decoder, and restarting decoding from the subsequent data of the previous process are known.
Of such methods, a method of extracting only required partial data from encoded image data having multi-resolution data and converting it into another encoded data has been conventionally proposed (see, e.g., U.S. Pat. No. 6,041,143).
Image data as a source in the above U.S. patent is encoded data which allows management of multi-resolution data using Wavelet transformation or Wavelet-like transformation. The above U.S. patent describes a method of extracting encoded data required to process data of a spatial region of user's choice from the source encoded data, and converting the extracted data into one independent encoded data. Note that partial encoded data extracted from the source encoded data correspond to code blocks of JPEG2000, and contain encoded data required to process the current request from the user.
With this method, when the client re-decodes encoded data sent from the server from the beginning, these fragmentary encoded data can be rebuilt into one encoded data file complying with JPEG2000 without any scheme like a cache.
However, since fragmentary encoded data are sent from the server side in the request order from the client side, a complicated process is required on the client side to convert the received encoded data into one encoded data complying with JPEG2000.
Also, fragmentary encoded data sent from the server may form a cache file of a unique format, and a JPEG2000 decoder on the client side may directly read this cache file. However, in this case, the JPEG2000 decoder requires a process for reading the unique cache file, resulting in a complicated process. In addition, a versatile JPEG2000 decoder cannot be used.
To avoid such problem, the following method may be used. That is, JPEG2000 encoded data are received for respective packets, and these packet data are cached. Upon passing encoded data to a decoder, zero length packet (to be abbreviated as “ZLP” hereinafter data are padded in place of non-received packet data to generate one encoded file complying with JPEG2000 together with the already received packet data. In this way, a JPEG2000 file that can be processed by a general JPEG2000 decoder can be generated without any complicated processes such as a rewrite process of a main header, and the like.
However, encoded data generated using all received packet data does not consider any display modes that the user wants in terms of the image size, image quality, image region, and the like requested by the user who uses the application on the client side. For this reason, a decoder that receives such encoded data often generates a display frame using a decode result generated from some of the received data, and generation of one JPEG2000 encoded data file using all cached encoded data often results in a wasteful process of the decoder itself.
Furthermore, as the size of encoded data to be cached increases, more copy processes of data are required upon generating encoded data to be passed to the decoder. This prolongs a time required until display on the client, thus posing a problem, i.e., performance drop. Especially, when the user issues image enlarge command, scroll command, and the like, the size of cache data, which are not directly required for display, increases, and the aforementioned problem occurs frequently.
Furthermore, the above U.S. patent suffers the following problem. That is, when the technique described in the above U.S. patent is directly applied to IIP as a communication via a network, encoded data of all code blocks of all layers are sent in response to each request from the client, and even encoded data that have already been received by the client are transmitted. The contents of the header field of encoded data to be transmitted are rewritten by information (e.g., image size, layer information, etc.) of the currently requested image data, and original information of encoded data managed by the server cannot be acquired.
With either of the aforementioned protocols, data transmitted from the server is independent encoded data. For this reason, the server can return such data regardless of the order of transmission data.
As a protocol for fragmentarily accessing and displaying a file encoded according to JPEG2000, JPEG2000 image coding system—Part 9: Interactivity tools, APIs and Protocols (to be abbreviated as JPIP hereinafter) has been examined. When JPEG2000 image data is fragmentarily transmitted to the client terminal using such protocol, the client terminal must cache received fragmentary encoded data in one or more files so as to decode the transmitted image data. This is because encoded data of each scalability of JPEG2000 is difference data from data one level lower than that scalability.
Upon caching fragmentarily received JPEG2000 data in files, a method of appending data to cache files in the order they are received is known. With this method, cache data can be easily written in. In this case, identification data, offset values, and data lengths of respective data are written so as to manage cached data.
The client that requests data using this JPIP can use the following two different requests in association with a return method of response data.
1) fullwindow: leaves the transmission order of response data to server's discretion, but requests to surely return all request data
2) progressive: allows to delete some request data, but requests to return data in order of scalability in the image quality direction (layer progression order)
Therefore, when the client wants to surely receive all requested data, it issues the fullwindow request. Hence, the server returns data according to the server's implementation policy regardless of the process on the client side.
However, packet data are not always sent from the server in the progression order of an original image in the server. When the client terminal launches a plurality of threads, which acquire packet data at a time from one JPEG2000 file, there is less chance that the packet data are sent in the progression order.
For this reason, when packet data are written in the cache file in the order they are received, access to information used to manage cached data becomes random, and the number of times of seek processes in the cache file becomes large upon writing cache data, thus taking much time.
Furthermore, when the client terminal, which receives such fragmentary JPEG2000 data, decodes and displays the received data, cache data must be rearranged to an order of data that the decoder can process before they are read out. For example, when a versatile JPEG2000 decoder is used, it reads out cache data along the JPEG2000 bitstream syntax to generate one bitstream. Therefore, when data are appended in the order they are received, the number of times of seek processes in the cache file also increases upon reading out cache data, thus taking much time.